Beyond MSER: Maximally Stable Regions using Tree of Shapes

Detection of local features which are distinctive, invariant and discriminative is used to construct compact image representations in many computer vision applications. Achieving robustness against viewpoint change motivated the development of affine invariant detectors responding to image gradient or contrast changes, edges or corners. We focus on the Maximally Stable Extremal Regions (MSER) detector [3] which responds to blobs of high contrast to produce affine invariant, distinctive arbitrary shaped regions. Exploiting the tree-based MSER computation algorithm [6], we replace the Min and Max-trees [7] in the algorithm with the Tree of Shapes [5], thus changing the pixel ordering used for region extraction. Min and Max-trees [7] represent the composition of complex images by encoding hierarchical relations of regions on various scales. The leaves of the Min-tree correspond to local image minima, while the inner nodes are (maximal) connected regions Rk at gray level k, such that all region pixel intensities f (p) are lower than k. The root region Rmax at the highest gray level covers the whole image. Distance between two nodes is their gray level difference: d(Rk,Rl) = |l− k|. The Max-tree is the dual hierarchy, corresponding to the Min-tree of an inverted image −I. Extremal regions used in MSER computation [3] correspond to the Min and Max-tree nodes. Minimal extremal regions Rk are connected regions in which all the elements on the outer boundary have strictly greater intensity than all the adjacent region elements, and are contained in the Min-tree. Similarly, the Max-tree comprises the maximal extremal regions. MSER computation is based on finding the local minima of the stability function q(·) for the extremal regions along the nested sequences: